# Face Recognition Algorithm ***Copyright © Quectel Wireless Solutions Co., Ltd. 2026. All rights reserved.*** --- This guide walks through building a desktop GUI face-recognition project from scratch on Quectel Pi M2 Debian, using the face-recognition algorithm SDK. When launched, the application opens a window on the device desktop and supports full-screen mode. The interface uses a three-column layout: - **Left column**: Algorithm modules and capabilities (face detection / 21-point landmarks / head pose / 1:N recognition) - **Center column**: Title, live display, and controls - **Right column**: Device status (face database count / RAM / ROM / CPU utilization) Two input modes are available, and faces can also be registered at runtime: - **Open Camera**: Open a USB camera, detect faces in real time, and compare them against the database using 1:N matching. - **Local Image**: Select a local image for offline face detection and recognition. - **Register Face**: Enter a name to save the largest face in the current frame that has `code == FACE_OK` and a valid feature to `facedb/registered/`, then add it to the in-memory database. It is restored automatically on the next launch. Registration is not reported as successful if saving fails. ## Algorithm Capabilities | **Item** | **Description** | | --- | --- | | Algorithm type | Face detection + 21-point landmarks + head pose + feature extraction and recognition | | Detection model | facedet_512_q_3576_enc.rknn (512² input, multi-scale face boxes) | | Landmark model | facelandmark_q_3576_enc.rknn (dynamic 21-point landmarks) | | Recognition model | facereg_l_q_3576_enc.rknn (512-dimensional face features) | | Input | Contiguous BGR memory (cv::Mat.data, rows × columns × 3) | | Output | One FaceResults_t per face (xywh box + confidence + yaw/pitch/roll + 21 landmarks + feature vector) | | Matching | 1:N cosine-similarity matching with a default threshold of 0.6; supports up to 10,000 database entries | **How results are presented in the demo**: - Face boxes are green for a database match and orange for an unmatched or unusable face. The UI does not display the detection confidence `conf` separately. - Toggle the 21 landmarks with the **[Landmarks]** button. - Head pose is shown below the box as `Y:yaw P:pitch R:roll`. - The name and match score appear above the box. Unmatched faces are labeled `Unknown`; faces with `code != FACE_OK` are labeled `low-quality`. `low-quality` is a generic demo label; consult the SDK status code for the specific reason. ## Prerequisites ### Obtain the Resources | **Name** | **Description** | | --- | --- | | face_RK3576 (algorithm SDK package) | Face-recognition package containing the public headers, shared libraries, third-party runtimes, models, sample images, and face database | | face_project (demo project) | Example project from this guide, including source code, Makefile, and three scripts for copying dependencies, building, and running | Key directories in the algorithm SDK package (`RK_FR_NPU_v1.1/rknn-2.3.2-linux`, referred to below as `face_RK3576`): ```bash rknn-2.3.2-linux/ ├── include/ │ ├── faceEngine.h # Public algorithm header (the only algorithm header required for integration) │ ├── authorization.h # Authorization definitions (module table / error codes) │ ├── opencv2/ # Bundled OpenCV headers │ └── rknn_api.h # RKNN API header (other RKNN headers are in the same directory) ├── lib/ # libfacerecognition.so and dependencies + libopencv_world.so ├── model/rknn/RK3576/ # Detection / landmark / recognition .rknn models ├── facedb/ # Face database (filenames are names) ├── image/ # Sample images └── main.cpp # Official minimal usage example ``` ### Hardware and System Requirements | **Item** | **Requirement** | | --- | --- | | Main board | Quectel Pi M2 | | System | Debian / Ubuntu aarch64 with a desktop environment (Wayland or X11) | | Peripheral | USB camera | | Connection | The device can be accessed from a PC using adb shell | ### Software Requirements The demo is built natively on Quectel Pi M2. Make sure the following software is installed: ```bash g++ --version # C++17 compiler make --version pkg-config --exists Qt5Widgets Qt5Gui Qt5Core && echo "Qt5 OK" # Qt5 (LGPL) for display v4l2-ctl --version # Enumerate and troubleshoot USB camera nodes ``` > **UI implementation**: The display layer uses the system-provided **Qt5**, so it displays directly on the desktop and supports full-screen mode. Capture, inference, and drawing use the OpenCV world library bundled with the SDK (`libopencv_world.so`, matched to `libfacerecognition.so`), isolated from the system OpenCV to avoid version conflicts. ## Project Structure ```bash face_project/ ├── src/ │ └── face_cam_demo.cpp # Main demo (Qt UI + inference worker thread + 1:N recognition) ├── include/ # Populated by setup: SDK public header faceEngine.h + bundled opencv2/rknn headers ├── lib/ # Populated by setup: libfacerecognition.so and dependencies + libopencv_world.so ├── models/ # Populated by setup: detection / landmark / recognition .rknn models ├── facedb/ # Populated by setup: face database (filenames are names) ├── images/ # Populated by setup: sample images ├── Makefile # Build rules ├── build.sh # One-command build ├── setup_from_sdk.sh # Copy dependencies from the SDK package into this project ├── run.sh # Configure environment variables and launch └── README.md # Documentation ``` `include/`, `lib/`, `models/`, `facedb/`, and `images/` are generated by `setup_from_sdk.sh` from the original SDK package. ## Core Code Walkthrough The demo uses the `FaceEngine` class exposed by the SDK's public `faceEngine.h` header. Its lifecycle is: construct → `init` (load models and authorize) → `set_param` (set thresholds) → `inference` (infer each frame) → `cosine_similarity` (1:N matching) → `uinit` (release resources). The source file is `src/face_cam_demo.cpp`. The following sections focus on the face-recognition calls. These snippets are from inside a function and require headers such as `faceEngine.h`, ````, and ````. ### Initialization ```cpp FaceEngine engine; char det_model[] = "models/facedet_512_q_3576_enc.rknn"; // Face detection char land_model[] = "models/facelandmark_q_3576_enc.rknn"; // 21-point landmarks char reg_model[] = "models/facereg_l_q_3576_enc.rknn"; // 512-dimensional feature extraction char license[512] = "face_license.ini"; if (const char* path = std::getenv("FACE_LICENSE_FILE")) { if (path[0]) std::snprintf(license, sizeof(license), "%s", path); } int module = 3; if (const char* value = std::getenv("FACE_MODULE")) { if (value[0]) module = std::atoi(value); } int app_id = 145; if (const char* value = std::getenv("FACE_APP_ID")) { if (value[0]) app_id = std::atoi(value); } char auth_buf[512] = {0}; if (const char* auth = std::getenv("FACE_AUTH_CODE")) { if (auth[0]) std::snprintf(auth_buf, sizeof(auth_buf), "%s", auth); } char* p_auth = auth_buf; FACE_RES_CODE rc = engine.init(det_model, land_model, reg_model, license, module, p_auth, app_id); engine.set_param(0.35f, 0.45f); int alg_status = engine.get_status(0); int auth_status = engine.get_status(1); std::printf("rc=%d alg_status=%d auth_status=%d\n", rc, alg_status, auth_status); ``` This snippet demonstrates environment-variable configuration. The demo executable also accepts the authorization code as its second command-line argument, which takes precedence over `FACE_AUTH_CODE`. Parameter descriptions: - **`init(det_model, land_model, reg_model, license, module, p_auth, app_id)`** loads all three models and performs authorization: - `det_model`: Face **detection** model (512²; outputs face boxes + confidence). - `land_model`: **Landmark** model (dynamic 21-point landmarks; also provides head pose). - `reg_model`: **Feature** model (outputs 512-dimensional face feature vectors for matching). - `license`: Path where the license file is stored. It is written after the first successful online authorization and can be used offline afterward. - `module`: Platform identifier; **pass`3`****for RK3576** (overridable with `FACE_MODULE`). - `p_auth` / `app_id`: Algorithm authorization code and authorization ID. - **`set_param(det_conf, nms)`**: `det_conf` is the face-detection confidence threshold (default `0.35`; increase it to reduce false positives); `nms` is the detection-box NMS IoU threshold (default `0.45`). - **`get_status(opt)`** queries status: `get_status(0)` returns algorithm initialization status (`0` means success); `get_status(1)` returns authorization status (`0` means fully authorized; other values are defined in `AuthErrCode_e`). ### Inference Call `inference` once per frame (from either the camera or a local image). The SDK performs detection → landmarks → pose → feature extraction and returns a complete result for each face. ```cpp std::vector results; // Arguments: BGR data pointer, rows (height), columns (width), channels (must be 3), output result vector FACE_RES_CODE inference_rc = engine.inference( frame.data, frame.rows, frame.cols, frame.channels(), results); if (inference_rc != FACE_OK) { std::printf("inference failed: %d\n", inference_rc); return; } for (const auto& r : results) { // r.code : 0 (FACE_OK) means usable; negative values indicate quality/pose/bounds issues, etc. // r.x, r.y, r.width, r.height : Face box (pixels in the original image) // r.conf : Face-detection confidence // r.yaw / r.pitch / r.roll : Head pose (yaw / pitch / roll) // r.key_pts : Coordinates of 21 landmarks // r.feature : 512-dimensional face feature vector (for 1:N matching) } ``` **Input requirements**: - Input must be non-empty and use **contiguous BGR memory**. A cropped ROI `cv::Mat` is not necessarily contiguous; check `isContinuous()` and call `clone()` if needed before passing it to the SDK. - `channels` must be `3`. Convert grayscale images to BGR with `cv::cvtColor` first. - The SDK performs model preprocessing internally. The caller must still ensure that the image format, channel count, memory continuity, and orientation are correct; do not repeat model normalization yourself. **Output structure`FaceResults_t`** (one per face): | **Field** | **Meaning** | | --- | --- | | code | 0 means the face is usable; negative values indicate that a feature could not be extracted due to quality, pose, bounds, or other issues. | | x / y / width / height | Face box in original-image pixels | | conf | Face-detection confidence | | yaw / pitch / roll | Head-pose angles (degrees) | | key_pts | 21 face-landmark points | | feature | This model produces a 512-dimensional face feature; before matching, check that code == FACE_OK and feature is non-empty. | ### 1:N Recognition Use `cosine_similarity` to compare the current face feature with the database (one feature per person) using cosine similarity. It returns the matched index and score. ```cpp // Database: run inference once for each image under facedb/, then use the filename without its extension as the name. std::pair m = engine.cosine_similarity(db_features, face.feature, 0.6f); // m.first >= 0 : Match at index m.first in the database; m.first == -1 : unknown face (Unknown) // m.second : Cosine similarity score (higher means more similar) ``` - The third argument is the **match threshold** (default `0.6`): a face is recognized only if its similarity is above the threshold and it is the top-1 match; otherwise it is `Unknown`. - At startup, each image in `facedb/` is read in turn. For each image, only the first face with `code == FACE_OK` and a non-empty feature is used; its name is the filename without the extension. Use base images containing exactly one target person to avoid assigning the wrong identity. - When **[Register Face]** is clicked, the largest usable face is selected. Its name and 512-dimensional feature are atomically saved to `facedb/registered/*.json`; after a successful save, the entry is added to the in-memory database. The camera image is not saved. On the next launch, base images are loaded first, followed by valid registered entries. Multiple samples for the same name are retained; existing entries are not overwritten. Use an English name or ID where possible because the current OpenCV image text does not support full Chinese names. - Registration JSON files use random filenames and owner-only read/write permissions. Run the program as the same user each time. Back up and protect biometric data in `facedb/registered/`, and do not commit it to a public repository. Re-register faces after changing the feature model. - A higher threshold is stricter (fewer false matches, more rejections); lower it as appropriate when device or lighting conditions differ significantly. ### Release ```cpp engine.uinit(); // Release detection / landmark / recognition models and authorization objects ``` ## Deployment and Run ### Transfer the Files Run the following from Windows PowerShell on the PC, in the directory containing the SDK package and project. Push the files to `/tmp` on the device first. ```bash adb devices adb push "RK_FR_NPU_v1.1/rknn-2.3.2-linux" /tmp/ adb push "face_project" /tmp/ ``` Then open a terminal in the Quectel Pi M2 desktop session and run the following as the regular user. These commands require `sudo`. If a directory with the same name already exists in the home directory, stop the program and back up the license and face database before updating; otherwise, the new directory could be moved inside the old one. ```bash sudo mv /tmp/rknn-2.3.2-linux /tmp/face_project "$HOME"/ sudo chown -R "$$USER:$$(id -gn)" "$$HOME/rknn-2.3.2-linux" "$$HOME/face_project" cd "$HOME/face_project" && chmod +x *.sh ``` ### Install the Build Environment and Tools Run on the device when it is connected to the network: ```bash sudo apt update sudo apt install -y build-essential pkg-config qtbase5-dev ``` ### Copy Dependencies `setup_from_sdk.sh` copies headers, shared libraries, models, sample images, and the face database from the SDK package into the corresponding project directories: ```bash cd "$HOME/face_project" sh setup_from_sdk.sh "$HOME/rknn-2.3.2-linux" # Omit the argument to use this default path. ``` ### Build ```bash sh build.sh #Output: /home/$USER/face_project/face_cam_demo ``` > `build.sh` invokes `make`: it uses the system Qt5 for the UI, then links the project-local `libfacerecognition.so` and bundled OpenCV world library to produce the executable. ### Run Run from the device's **desktop session** (the Qt window requires a desktop environment): ```bash cd "$HOME/face_project" ./run.sh ``` Once `./run.sh` opens the UI, click **Open Camera** to start capture. You can also specify a device node based on the output of `v4l2-ctl --list-devices`, for example `./run.sh /dev/video73`; the node number is only an example. If the specified node cannot be opened, the program still attempts to detect a USB capture node automatically. > We recommend launching from a terminal in the device's desktop session. If using `adb shell`, switch to the desktop login user and make sure `XDG_RUNTIME_DIR`, `WAYLAND_DISPLAY`, or `DISPLAY`/`XAUTHORITY` matches the actual desktop session. The script's display-platform detection cannot guarantee that root or another user can connect to the desktop. ### Use the UI The window has three columns: algorithm modules on the left, display and controls in the center, and device status on the right. | **Control** | **Description** | | --- | --- | | **[Open Camera]** | Open/close the USB camera for real-time detection and 1:N recognition; closing it clears the display. | | **[Local Image]** | Open a file picker and run offline recognition on a selected image. | | **[Register Face]** | Freeze the current frame and prompt for a name; register the largest face so it can be recognized afterward. | | **[Landmarks]** | Toggle display of the 21 face landmarks. | | **[Full Screen]** | Toggle between full-screen and windowed mode. | | **[Quit]** | Close the program. | | Keyboard shortcuts | Esc/Q to quit; F for full-screen mode. | The **device status panel** on the right refreshes once per second: | **Metric** | **Description** | | --- | --- | | Face DB | Current number of faces in the database, including runtime registrations | | RAM | Used percentage and used/total memory (MB) | | ROM | Root filesystem usage percentage and used/total capacity (GB) | | CPU | CPU utilization percentage | ## Authorization The license file and application configuration must match the device and the scope of the license: - Supply the authorization code through `FACE_AUTH_CODE` (already exported by `run.sh`; replace its value with your own code). - `FACE_MODULE` defaults to `3` (RK3576; see `include/authorization.h`). - `FACE_APP_ID` currently defaults to `145`, corresponding to `CV_SET_APPID`. - `FACE_LICENSE_FILE` defaults to the absolute path `$ROOT/face_license.ini` inside the project. If no valid local license exists, the SDK attempts to obtain one online. After successful authorization, it saves a device-bound file and prefers that local file on subsequent launches. Keep the file and make sure the runtime user can write to the project directory. Do not distribute authorization codes, license files, or face databases with a public project. - A valid local license may allow authorization to succeed without an online request using the newly supplied code. When testing a new code, distinguish the test from reuse of a local license; do not infer success solely from the file's existence, its non-empty contents, or the UI opening. Status queries: `get_status(0)` returns algorithm initialization status (`0` means success); `get_status(1)` returns authorization status (`0` means success; other values are defined in `AuthErrCode_e`). ## Demo Video
face-demo
## Troubleshooting | **Issue** | **Possible cause** | **Solution** | | --- | --- | --- | | Initialization fails (alg_status != 0) | Model is missing or initialization failed | Copy dependencies first, verify the paths to all three models, and use the SDK return code to investigate. | | Authorization fails (auth_status != 0) | License, application ID, device binding, or resource scope does not match | Check FACE_AUTH_CODE, FACE_APP_ID, and FACE_MODULE; contact the licensing provider. Do not change the module number without confirmation. | | auth_status=100024 | The requested FaceRecognition resource is outside the license scope | Ask the licensing provider to confirm the application and resource entitlements. | | Registration cannot be saved or is not restored after restart | Directory is not writable, disk is full, record is corrupt, or the runtime user/model changed | Check facedb/registered/, file ownership, and runtime logs; launch from the same project directory. Re-register after changing the feature model. | | Window does not appear | Not running in a desktop session / QT_QPA_PLATFORM does not match | Run in the device's desktop session, or let run.sh detect Wayland/xcb automatically. | | Camera cannot be opened | USB camera is disconnected / specified node is wrong | Automatic mode probes for USB capture nodes. Use v4l2-ctl --list-devices to check, then run ./run.sh /dev/videoN. | | Link or runtime error: .so not found | LD_LIBRARY_PATH is not set | Launch with run.sh (which configures it), or export the project lib path manually. | | Face is labeled low-quality | A face box was detected, but pose, size, bounds, or quality did not meet the requirements for feature extraction | Face the camera directly and keep the face clear and fully in frame; adjust the detection confidence in set_param if needed. | ## Support and Contributions For help, submit a technical question to the [Quectel official forum](), where the Quectel technical support team can assist. Project repository: [ https://github.com/Quectel-Pi/demo-edge-algorithms ]() Issues and pull requests for improvements are welcome.